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ABSTRACT 



Covariance and correlation structure analytic techniques can 
be used to test whether a specified correlation structure is an adequate 
model of the population correlation structure. These procedures include: (1) 

normal theory (NT) and asymptotically distribution free (ADF) covariance 
structure analysis techniques; and (2) NT and ADF correlation structure 
analysis techniques. This paper discusses Monte Carlo results on the Type I 
error control of correlation structure analytic techniques for tests of 
correlation pattern hypotheses under conditions of multivariate nonnormality. 
The results show the clear nonrobustness of normal theory correlation 
structure analysis procedures under conditions of nonnormality when testing 
the correlation pattern hypotheses such as the simplex or circumplex, but 
less so when testing the diagonal or block-diagonal correlation pattern 
hypotheses. This paper further demonstrates how improved Type I error control 
can be obtained by adopting asymptotically distribution free correlation 
structure analysis procedures. Three appendixes present population 
correlation matrices and model matrices, a discussion of distribution types, 
and a table of empirical Type I error rates. (Contains 6 tables, 8 figures, 
and 30 references.) (Author/SLD) 
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Abstract. Covariance and correlation structure analytic techniques can be used to test whether a specified correlation 
structure is an adequate model of the population correlation structure. These procedures include (i) normal theory 
(NT) and asymptotically distribution free (ADF) covariance structure analysis techniques, and (ii) NT and ADF 
correlation structure analysis techniques. This paper discusses Monte Carlo results on the Type I error control of 
correlation structure analytic techniques for tests of correlation pattern hypotheses under conditions of multivariate 
nonnormality. The results show the clear nonrobustness of normal theory correlation structure analysis procedures 
under conditions of nonnormality when testing the correlation pattern hypotheses such as the simplex or circumplex, 
but less so when testing the diagonal or block-diagonal correlation pattern hypotheses. This paper further 
demonstrates how improved Type I error control can be obtained by adopting asymptotically distribution free 
correlation structure analysis procedures. 



Subject descriptors: Covariance structure analysis, correlation structure analysis, normal theory, asymptotically 
distribution ffee, quadratic form statistics, Fisher transform, robustness, multivariate normality, multivariate 
nonnormality, Type I error. 



Introduction 



Often researchers engaged in nonexperimental research find it useful to describe the patterns of association 
in their data in terms of the structure of the correlation or covariance matrix. For a discussion of structure analysis 
techniques, two types of hypotheses can be distinguished. A structure hypothesis refers to any hypothesis that 
prescribes values for, or relations between the elements of a given matrix. A pattern hypothesis specifies certain 
groups of elements in a matrix to be equal to each other, and/or to a specified numerical value (McDonald, 1974, 
1975; Steiger, 1980a). 

Familiar patterns of association 

Certain examples of covariance and correlation patterns are well known in education, and the behavioral 
and social sciences. Familiar correlation patterns include diagonal, block-diagonal, circumplex, and simplex patterns. 
Other interesting hypotheses, such as, those of constancy of covariance and correlation matrices over time can also 
be expressed in terms of tests of certain patterns of association. 

Diagonal patterns of association occur when there is a lack of pairwise association in a given set of 
variables; the question of the diagonality of the pattern of association addresses whether the observed departures in 
the off-diagonal elements of the matrix from zero reflect true departures from zero or whether the observed 
departures from zero are simply resultant from sampling variation. By addressing this question, the researcher tries to 
determine if any of the variables are related. If after appropriate statistical analysis, the researcher identifies the 
probability that the observed pattern of association originated from a population where the variables are uncorrelated 
is low, many other structural questions can be addressed regarding the nature of the pattern of association. 

Block-diagonal patterns of association are observed when there is a lack of association between different 
sets of variables; the question of whether the pattern of association is block-diagonal is relevant when examining the 
association between several sets of variables and addresses whether the observed intercorrelations between sets of 
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variables reflect true setwise intercorrelation or whether the departures from zero are resultant from sampling 
variation. Consider an example where a researcher wishes to explore the relationship among three sets of variables. 
Imagine these sets of variables are individual personality, academic achievement and job success variables. The 
researcher can be interested in a number of different questions. But a first overall question is to determine whether 
there is any relationship between the sets of variables. For this, the data are analyzed to determine whether a block- 
diagonal model is an appropriate characterization of the pattern of association. 

Simplex and circumplex models also have wide application. Guttman (1954) introduced the terms simplex 
and circumplex to refer to inequality patterns observed in correlation matrices for linearly or circularly ordered tests. 
Simplex and circumplex patterns can arise when equally spaced temporal or spatial variables are similarly correlated. 
A number of recent papers have applies the simplex model in characterizing patterns of association; these papers 
- include Raykov and Stankov’s (1993) examination of task complexity, and Marsh’s (1993) exploration of the 
stability of individual differences in multiwave panel studies. 

Procedures for the confirmatory analysis of patterns of association 

A wide variety of statistical procedures can be used for confirmatory analysis of patterns of association . 
Many of these procedures fit into two general classes of structure analytic techniques. These procedures include 
covariance structure analysis techniques and correlation structure analysis techniques, where covariance and 
correlation structure techniques can be distinguished from each other in terms of the theory on which they are based. 
In the present paper, covariance structure analytic procedures refer to procedures based on distribution theory for 
covariances, whereas, correlation structure analytic procedures refer to procedures based on distribution theory for 
correlations. 

- When researchers are interested in models that are not scale invariant, they are restricted to using 

covariance structure analytic procedures; however, if they are interested in scale invariant models and have specific 
hypotheses about the association among the observed variables, they can use either covariance structure analytic 
techniques or correlation structure analytic techniques to address their data analytic question. However, historically, 
applied researchers have tended to use covariance structure analysis techniques over correlation structure analysis 
techniques, even when analyzing the correlation pattern among the observed variables. This paper focuses on the use 
of correlation structure analysis techniques based on correlation distribution theory. 

Background 

Correlation structure analysis techniques test the null hypothesis that the p x p population correlation 
matrix, P , can be expressed as a matrix valued function of t-dimensional parameter vector ^ , using asymptotic 
correlation distribution theory. Let r = Vec(R) and p =Vec(P) y z(r) = {z(r / )} and z(p) = {z(p, )}, where z(x) is 
the Fisher z-transform of x . 

Over the decades, various approaches have been proposed to test the adequacy of hypothesized scale 
invariant models and correlation models. These include normal theory procedures and asymptotically distribution 
free procedures proposed for use when the distributional assumption of multivariate normality which underlies some 
of the correlation structure analysis techniques is not necessarily tenable or one that the researcher wants to make. 

Work by Micceri (1989) highlights the prevalence of data in education and the behavioral sciences with 
characteristics which depart from those of multivariate normal distributions. Micceri examined data from 440 large- 
sample achievement, criterion mastery, and psychometric measures. On the basis of his results, he concluded that 
very few of the distributions were “even reasonably close approximations to the Gaussian.... [and] one should 
probably heed Geary’s (1947) caveat and pretend that “normality is a myth; there never was, and never will be, a 
normal distribution” ” (p. 161). Given that univariate normality is a necessary condition for multivariate normality, 
the conclusion is that in few of these data sets could the condition of multivariate normality be said to hold. As such, 
the use of normal theory techniques would be inappropriate for use if/when normal theory are not robust to violations 
of the distributional assumptions of multivariate normality, and use of the asymptotically distribution free procedures 
may be preferred. However, to date, few papers have discussed the performance of these procedures under 
conditions of multivariate nonnormality. 

Normal theory tests 

An array of normal theory tests of correlation structure can be obtained using either quadratic form tests 
based on correlation distribution theory, or quadratic form tests based on distribution theory for the Fisher z- 
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transforms of the correlations (Mooijaart, 1985; Steiger, 1980a,b; Steiger & Browne, 1984; Steiger & Hakstian, 
1982, 1984). Substituting estimates generated by either the method of maximum likelihood or normal theory 
generalized least squares yields alternative tests of correlation structure that under true null hypotheses are 
asymptotically chi-square with g = p(p + l) / 2- q degrees of freedom where q-t + p . 

This paper considers the normal theory quadratic form statistics based on generalized least squares 
estimates, QfttR anc * QNtZ * QNtR = ~l)(r-PGisY'¥Ls( r ~PGls) uses the inverse of the normal theory 
estimate of the covariance matrix of the correlation coefficients [J , and the normal theory quadratic form Fisher 
transform-based statistic Qj\/tZ = (N -3)(z(r)-z(p Ciy )) , 'F^ (z(r)-z(Pcfr)) uses the inverse of the normal 

theory estimate of the covariance matrix of the Fisher z-transform of correlation coefficients (Steiger, 

1980a,b; Steiger & Hakstian, 1982). Few comprehensive simulation studies have been conducted on the application 
of this procedure under conditions of multivariate nonnormality (Fouladi & Steiger, 1995; Fouladi & Steiger, in 
press; Fouladi, in press). 

Asymptotically distribution free tests 

Asymptotically distribution free quadratic form statistics based on correlation distribution theory and 
distribution theory for Fisher z-transforms are also available (Mooijaart, 1985, Steiger & Hakstian, 1982, 1984). This 
paper considers the versions of the asymptotically distribution free theory quadratic form statistics using generalized 
least squares estimates, QAdfR anc * QAdfZ * which under a true null hypothesis are approximately chi-square with 

g degrees of freedom. 

Using the correlation version of the quadratic form statistic in Steiger (1980a,b), substitution of the 
asymptotically distribution free estimate of the covariance matrix of correlation coefficients computed using ordinary 
-least squares estimates of the correlation coefficients yields the asymptotically distribution free test of correlation 

structure, Q^dfR = (N - l)( r -*pGiy)^A4/7?( r “'PGiy) * where is the inverse of the asymptotically distribution 

free estimate of the covariance matrix of the correlation coefficients. 

Using the Fisher z-transform version of the quadratic form statistic in Steiger (1980a,b), substitution of the 
asymptotically distribution free estimate of the covariance matrix of the Fisher z-transform of the correlation 
coefficients yields alternative asymptotically distribution free tests of correlation structure, 

2 A 4 ^Z = (^ _ 3)(z(r)-z(p G / i )) , 'P4j / y Z (z(r)-z(p G i r )) , where is the inverse of the asymptotically 

distribution free estimate of the covariance matrix of the Fisher z-transform of the correlation coefficients. 

Relevant Monte Carlo Research 



To date, very few comprehensive comparative studies have been published on the performance of these 
general correlation structure analysis techniques under conditions of multivariate normality and nonnormality. 
Almost all of these studies have documented the performance profiles of normal theory techniques; few studies have 
documented the performance of the asymptotically distribution free correlation structure analytic techniques. 

Steiger (1980a) compared the performance of normal theory statistics for tests of correlation pattern 
hypotheses under conditions of multivariate normality. Steiger showed that when the data are drawn from a 
multivariate normal distribution, the normal theory quadratic form Fisher z-transform based statistics have notably 
superior Type I error rate performance at smaller sample sizes. Fouladi (dissertation 1996, 1997) examined both 
normal theory and asymptotically distribution free quadratic form procedures and found that even though none of the 
statistics had maintained strict control of Type I error, not only did the normal theory procedures outperform the 
asymptotically distribution free procedures under multivariate normality, they performed quite well across nearly all 
sample sizes. 

Fouladi and Steiger (1995, in press) show that normal theory correlation structure analysis techniques do 
not perform well under conditions of multivariate nonnormality. So far, few empirical studies have documented the 
performance of the asymptotically distribution free quadratic form correlation structure analysis procedures under 
conditions of multivariate nonnormality. 
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The Purpose Of This Study 

Many structure analysis experts agree that normal theory covariance structure analysis procedures should be 
used with caution under conditions of multivariate nonnormality); though some do provide conditions for the 
robustness of normal theory covariance structure analysis techniques (c.f., Fouladi, in press). In general, experts 
recommend the use of alternative procedures, though not the asymptotically distribution free generalized least 
squares covariance structure analysis procedure which has been shown to have poor performance characteristics 
under all but the largest sample sizes (Chou, Bender, & Satorra, 1991; Curran, West, & Finch, 1996; Henly, 1993; 
Hu, Bentleri & Kano, 1992), and is the covariance version of the asymptotically distribution free correlation 
structure analysis procedure in the present paper. 

Similar cautionary statements have been made for the use of correlation structure analysis procedures 
(Fouladi & Steiger, 1995, in press), however, the performance characteristics of the general asymptotically 
distribution free correlation structure analysis procedures have not been widely documented. 

At issue is the question of the relative performance of the normal theory and the asymptotically distribution 
free correlation structure analysis procedures under conditions of multivariate nonnormality. Fouladi (dissertation, 
1996) examined normal theory and asymptotically distribution free covariance and correlation structure analysis 
procedures under conditions of multivariate nonnormality. The performance of these techniques was assessed across 
different types of models, number of variables, sample sizes, marginal skew and kurtosis, and nominal alpha levels. 
In this paper, I report and discuss my results on the Type I error control of correlation structure analytic techniques 
for tests of correlation pattern hypotheses under conditions of multivariate nonnormality. Results are also examined 
using guidelines acceptable Type I error control recommended by Bradley (1978) and Robey and Barcikowski 
(1992). 



Methods 



A series of Monte Carlo simulation experiments were conducted in order to examine the error rate control 
of the different correlation structure analysis test proceudres. I wrote a stand-alone FORTRAN computer program 
implementing normal theory and asymptotically distribution free correlation structure analysis techniques. 
Programming accuracy checks were done with MULTICORR (Steiger, 1979) and Mathematica (Wolfram, 1996). 

For the examination of Type I error control, data was generated from multivariate populations with 
specified univariate and bivariate moments. The populations were varied along three dimensions: (a) number of 
variables, p , (b) distributional characteristics of the variables, and (c) correlation pattern among the variables. 

The populations were p- variate, where levels of p included 2, 6, and 12. Each of the variables in the p- 
variate population had means equal to zero, and variances equal to 1. The distributional shape of the variables was 
varied, where levels of kurtosis, Ku, included -1, 0, 1, 3, 6, 25, and levels of skew, Sk, included 0, 1, 2. The 
correlational model, P, among the variables was varied such that the population correlation matrix was either 
diagonal, block-diagonal where each block is composed of p! 2 variables, simplex, symmetric or circular/circumplex. 
Appendix A details the population correlation matrices examined in the present study. Sample matrices from all 
model conditions were not generated: for p- 2, only the diagonal was simulated; for p=6 all model types were 
simulated; and for p=\2 y only diagonal and circumplex models were simulated. 

The methods of Fleishman (1978) and Vale and Maurelli (1983) were used to generate independent 
identically distributed observations from specified multivariate nonnormal distributions, with known correlation 
structure, marginal skew and kurtosis. With this method, not all combinations of skew and kurtosis are possible; 
Appendix B details the combinations of kurtosis and skew examined in the present study. 

Tanaka (1987) suggested that the ratio of sample size to number of parameters in the covariance structure 
model is one way of examining whether one’s sample size is “big enough”. Sample correlation matrices were 
generated as various sample sizes, N \2q ,4q , 10^,20^, and 50^ where q =p(p+l)/2-g=p+t. 

Hypotheses were tested at two levels of nominal Type I error: a =.05 and .01. For each sample correlation 
matrix, the available correlation structure analysis statistics for each test of correlation pattern were calculated; the 
decisions for the tests were recorded at each of the nominal levels. 

Experiments under each condition were replicated 5000 times. 
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Results 



Under each condition, the rejection frequency for each statistic was observed. For each condition, the 
number of rejections obtained for each correlation pattern test was tabulated and transformed into proportion 
rejected. These results are provided in Appendix C. Some of the procedures could not be used under a few of the 
experimental conditions with extremely small sizes; under those circumstances, no rejection rates are reported or 
analyzed. And in one set of conditions (p= 6, circumplex, univariate kurtoses=25 and skew=3) anomalous results 
were observed; these results are reported but not included in the summary analyses. 

Overall Type I error control 

Table 1 details empirical rejection rate summary statistics for each of the procedures at each level of 
nominal alpha. Mean rejection rates results suggest similar patterns of Type I error control at the .05 and the .01 
level. The mean rejection rates indicate that overall AdfR showed conservative bias, and AdfZ \ NtR , and NtZ showed 
liberal bias. Overall across all the conditions, AdfR showed the least bias, followed by AdfZ , then NtR, and NtZ 
Results show that there is a radical difference in the variability of the rejection rates for the four procedures, with 
AdfR showing the least variability of the four procedures. 

Tests on empirical rejection rates 

A one-way repeated measures analysis of variance was conducted on the empirical rejection rates using a 
multivariate approach; results indicated that overall there was a significant difference between the average empirical 
rejection rates of the four procedures (pc.OOl). 

Multivariate and univariate analyses of variance were conducted on the observed differences between 
empirical alpha and nominal alpha, B = ^Empirical ~ ^Nominal > f° r each of the procedures, to determine whether 

there the observed bias in the Type I error rates of the procedures under study was within sampling error of 0. 
Results showed that there was a significant difference between the observed bias and 0 overall (p<.001), and for each 
of the procedures (p<.001). 

Null-consistent chi-square goodness of fit values based oh the normal approximation to the binomial were 
used to assess the departure of the empirical Type I error rate from the nominal level. Chi-square values were 
computed for each test statistic. The overall fit values revealed that none of the test statistics can be said to 
consistently provide overall control the Type I error rate at the nominal level (p<.001). 

Judgments on empirical rejection rates 

Bradley (1978) and Robey and Barcikowski (1992) provided guidelines for judgments on the adequacy of 
Type I error control of procedures. In 1978, Bradley wrote a paper in which he asserted that many researchers are 
unreasonably generous when defining acceptable departures of empirical alpha from the nominal level. Bradley held 
that the departure of empirical alpha from the nominal level was “negligible” if empirical alpha was within a ±~a 

according to a ‘fairly stringent criterion’, and a±\a according to the “most liberal criterion that [he] was able to 

take seriously” which in the remainder of his article he referred to as the ‘liberal criterion’. Robey and Barcikowski 
(1992) supplemented the guidelines provided by Bradley for defining acceptable departures from the nominal level, 
providing an ‘intermediate criterion’ of a±j(X , and a ‘very liberal criterion’ of a±^a. The Bradley-Robey- 

Barcikowski guidelines for acceptable departure of empirical Type I error rates from the nominal level are hereafter 
referred to as the BRB criteria. 

Table 2 details the lower and upper limits of the acceptable levels of Type I error control according to the 4 
BRB guidelines with nominal alpha of .05 and .01. From these limits and the summary statistics provided in Table 1 
on the minimum and maximum observed Type I error rates, it is clear that no procedure provides consistent control 
of empirical Type I error rates within even the most liberal of the BRB guidelines, a±-a, across all of the 
conditions examined in this study. 

Type I error control as a function of nominal alpha, p. model type, sample size, and distribution type 

A five-way factorial multivariate analysis of variance was conducted to determine the influence of model 
type, distribution type, sample size, and nominal alpha on observed bias ( Z? = a Empirical ~ a Nominal)- Only first 
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order and second order effects were included in the analyses. With the exception of p x distribution type (Pillai, 
p=.940), all multivariate tests of main effects and two-way interaction effects yielded pc.OOl. Univariate analyses 
showed that over 90% of the variance in the departures of empirical rejection rates from the nominal level were 
explained by first order and second order effects. Obtained R-squared values were .975, .985, .955, and .907 for NtR , 
NtZ , AdfR , and AdfZ respectively; corresponding adjusted R-squared values were .966, .980, .941, and .876. Table 3 
details results from the univariate analyses. 

** •* Judgement based on chi-square/df values 

Chi-square goodness of fit values to assess the departure of the empirical Type I error rate from the nominal 
level were computed for each test statistic as a function of nominal alpha, p y model type, sample size, and 
“ distribution type. Since the degrees of freedom for the chi-square tests varied across conditions. In order to obtain a 
measure of Type I error control which would permit comparison among the procedures chi-square values were 
divided by their corresponding degrees of freedom; these values are reported in Table 4. The magnitude of the 
measures of Type I error control (chi-square/df) reveal that AdfR has the best overall Type I error control of all the 
structure analytic procedures. 

As a function of nominal alpha, the top procedure is AdfR for both nominal alpha of .05 and .01; and in 
general Type Terror rate control is better at the .05 level than at the .01 level for all the procedures except AdfR. 

Expressed as a function of p, the top procedures are (a) NtZ for p= 2, (b) AdfR for p = 6 and 12; and in 
general Type I error control is better at smaller levels of p for all the procedures except for AdfR which has relatively 
stable control across levels of p. 

'* - Performance considered as a function of model, the top procedures are (a) NtR and AdfZ for the diagonal 

- model (b) NtR and AdfR for the block-diagonal model, (b) AdfR for the simplex and the circumplex models; in 
general for any model in which variables are correlated, the Type I error control is considerably worse than the 
models in which variables are uncorrelated for all procedures except for AdfR which has relatively stable control 
across different model types. 

For different levels of sample size, the top procedure is AdfR.. Over all conditions Type I error control 
worsened for NtR and NtZ for increasing sample size, whereas AdfR and AdfZ showed improved Type I error control 
as sample size increased. 

The top procedures for different types of distributions ( K,S) are (a) NtR and NtZ for distributions with 
homogeneous marginals (-1,0), (b) R, Z, and AdfR for homogeneous marginals (1,0), (d) AdfR for homogeneous 
marginals (1,1), (3,0), (3,1), (6,0), (6,1), (6,2), (25,0), (25,1), (25,2), (25,3), (0 AdfR and NtR for the one 
heterogeneous marginals condition examined; in general Type I error control worsened for increasing leptokurtosis 
and skew except for AdfR which showed relatively stable Type I error control across distribution types. 

Judgments based on Bradley-Robey-Barcikowski critieria 

Departures of empirical rejection rates from the nominal level B = ^Empirical - a Nominai were obtained for 

each of the procedures, and when expressed as percentage of the nominal level yield percent bias 
B% = 1005/ ^Nominal • Minimum and maximum percent bias results for the different procedures across all 
simulated distributions types are reported as a function of the model, p, sample size, and nominal alpha in Table 5. 
Select results are reported as a function of the model, sample size, and distribution type in Table 6. As is known from 
the overall minimum and maximum empirical rejection rates at the different levels of alpha, none of the procedures 
can be described as providing consistent control of empirical rejection rates within even the most liberal of the BRB 
criteria ( a ±j(X ); however these tables permit further exploration of empirical Type I error control as a function of 
model, p, sample size, nominal alpha, and distribution type. 

Type I error control as a function of p, model type, sample size, and nominal alpha 

Table 5 details the minimum and maximum percent bias results for the different procedures across all 
simulated distributions types, also reported are the percentage of distribution conditions in which empirical rejection 
rates fell within the most liberal of the BRB guidelines for acceptable Type I error control a±|a. 

For tests of the p = 2 diagonal model at the .05 level, NtZ provides control at the liberal BRB level across all 
levels of N:q and distribution types, and AdfZ functionally controls empirical Type I error rates within the BRB 
criteria across all levels of N: q. For consistent control within any of the BRB criteria by the other procedures at the 
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.05 nominal level, N:q of at least (a) 4 was needed for NtR, (b) 10 ioi AdfR. At the .01 level, no minimum N:q can be 
specified for any of the procedures beyond which they provide consistent control of Type I error within even the 
most liberal of the BRB guidelines. 

For tests of the p= 6 diagonal model at the .05 level, NtR functionally controls empirical Type I error rates 
within the BRB criteria across all levels of N:q. For control within any of the BRB criteria by the other procedures at 
the .05 nominal level, N:q of at least (a) 20 was needed for AdfR and AdfZ, (b) and 50 for NtZ. At the .01 level, no 
minimum N:q can be specified for any of the procedures beyond which they provide consistent control of Type I 
error within^ven the most liberal of the BRB guidelines. 

For tests of the p - 12 diagonal model at the .05 level, no procedure provides control within the BRB criteria 
across all levels of N: q. For control within any of the BRB criteria at the .05 nominal level, N:q of at least (a) 20 was 
* needed for AdfR and AdfZ, (c) 50 for NtR and NtZ. At the .01 level, no minimum N:q can be specified for any of the 
procedures beyond which they provide consistent control of Type I error within even the most liberal of the BRB 
guidelines. 

For tests of the p = 6 block-diagonal model at the .05 or .01 level, no procedure provides control within the 
_ BRB criteria across all levels of N:q. For control within any of the BRB criteria at the .05 nominal level, N:q of at 
least (a) 20 was needed for AdfR and AdfZ, and (b) 50 for NtR and NtZ. At the .01 level, no minimum N:q can be 
specified for any of the procedures beyond which they provide consistent control of Type I error within even the 
most liberal of the BRB guidelines. 

For tests of the p - 6 simplex model, no procedure provides control within the BRB criteria across all levels 
of N:q at the .05 or .the .01 level of nominal alpha. For control within any of the BRB criteria at the .05 nominal 
- level, N:q of at least (a) 10 was needed for AdfR, (b) 50 for AdfZ. For tests at the .01 level, no procedure provides 
_ control within the BRB criteria across all levels of N:q. For control within any of the BRB criteria at the .01 nominal 
level, N:q of at least (a) 50 was needed for AdfR, and AdfZ. 

For tests of the p - 6 circumplex model, no procedure provides control within the BRB criteria either at the 
.05 level or the .01 level of nominal alpha across all levels of N:q. For control within any of the BRB criteria at the 
.05 nominal level, N:q of at least (a) 10 was needed for AdfR,. For control within any of the BRB criteria at the .01 
nominal level, N:q of 50 was needed for AdfR. At the .01 level, no minimum N:q can be specified for any of the 
procedures beyond which they provide consistent control of Type I error within even the most liberal of the BRB 
guidelines. 

For tests of the /?=12 circumplex model at the .05 or .01 level, no procedure provides control within the 
BRB criteria across all levels of N:q. For control within any of the BRB criteria at the .05 or .01 level, N:q of 20 or 
more was needed for AdfR . None of the other procedures control the Type I error rate within the BRB bounds of 
acceptable control of Type I error. 

Boxplots of the empirical rejection rates as a function of sample size are provided in Figures 1-4 for select 
conditions (Note: Figures 1-4 are not on the scale). Figures 1 and 2 illustrate the performance of NtR and NtZ as a 
function of sample size for the diagonal and block-diagonal models across all distribution types, and depict 
reasonable control of Type I error for both NtR and NtZ. Figures 3 and 4 illustrate the performance of AdR and AdfZ 
as a function of sample size across all four models and distribution types. Figure 3 shows that though AdfR is clearly 
conservative at lower levels of sample size, the empirical rejection rates show rapid convergence to the nominal level 
as sample size increases. Figure 4 shows that, though AdfZ clearly does not control Type I error close to the nominal 
level across all conditions, the empirical rejection rates show rapid convergence to the nominal level as sample size 
increases. 

Type I error control as a function of model type, sample size, and distribution type 

Table 6 details the performance of the procedures (B%) as a function of marginal kurtosis ( Ku ) and marginal 
skew ( Sk ) for each of the models with p - 6 at nominal alpha of .05, for N:q of 50, 20, 10, 4, and 2, respectively; due 
to space considerations, percent bias (B % ) results are only displayed at the .05 level; results at the .01 level are not 
displayed. Examination of the pattern of positive and negative departures of empirical Type I error from the nominal 
level reveal that the normal theory procedures NtR and NtZ tend to show liberal bias for the simplex and circumplex 
models. AdfR tends to be conservative; and AdfZ tends to be liberal except for diagonal and block-diagonal models. 
Furthermore, the results show that for distributions with marginal kurtosis and skew (K,S) of (1,0), and (-1,0), all of 
the procedures meet the BRB criteria for acceptable Type I error control except (a) AdfR and AdfZ when N:q= 4, (d) 
Z, AdfR, and AdfZ when N:q= 2; the findings reveal that the procedures with the best overall small sample 
performance for these distributional conditions is NtR. The results for other distributional conditions show that 
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normal theory techniques do not in general control Type I error; for other distribution types, the preferred procedures 
are any of AdfR or AdfZ depending on sample size and model conditions. Examination of the percentage departures 
of empirical Type I error rates from the nominal level as a function of model and distribution type at different levels 
of N:q further reveal (a) the asymptotic robustness of normal theory procedures for the diagonal and block-diagonal 
model, and (b) the robustness of normal theory procedures for the simplex and circumplex models under 
distributional conditions with marginal kurtoses of -1 or 1 and marginal skew of 0 at the nominal level of .05. Results 
from the .01 level reveal a similar pattern, except that robustness of normal theory procedures for the simplex and 
cireumplex models is also observed under distributional conditions with marginal kurtoses of -1, 1 or 3 and marginal 
skew of 0 or 1. 

Figures 5a-d depict the influence of model type and sample size on the empirical rejection rates of the NtR, 
NtZ , AdfR v and AdfZ\ again due to space considerations results are only displayed for select conditions, namely .05 
nominal level, p=6. Figures 5a and 5b illustrate how the empirical rejection rates of NtR and NtZ are extremely 
different for the different model types. The empirical rejection rates of NtR and NtZ vary relatively little for the 
diagonal and block-diagonal models, in comparison with the variation observed for the simplex and circumplex 
models where they vary both as a function of sample size and distribution type; under the simplex and circumplex 
model Type I error rates are observed increasing as a function of increasing departure from normality, and this 
becomes more severe as sample size increases. In contrast. Figures 5c and 5d, which are portrayed on the same scale 
as Figures 5a and 5b, illustrate very different surfaces for the empirical rejection rates of AdfR and AdfZ. Figure 5c 
illustrates how the empirical rejection rates of AdfR vary relatively little as a function of model type, sample size, or 
distribution type. Figure 5d illustrates how the empirical rejection rates of AdfZ varies as a function of model type, 
with the empirical rejection rates varying relatively little for the diagonal model, more so for the block-diagonal, and 
much more so for the simplex and circumplex models; for the block-diagonal model, simplex, and circumplex 
models, the greatest variation in the empirical rejection rates are seen at the lower levels of sample size, with the 
increases in the Type I error rates becoming more severe as a function of increasing departure from normality. 

Discussion and conclusions 



The distributional assumption of multivariate normality underlies many of the techniques used in the 
statistical analysis of multivariate data. Clearly, the issue of the distributional characteristics of one’s data is 
unimportant if statistics are robust to violations of distributional assumptions; however, considerable research 
suggests that parametric statistics frequently exhibit either relative or absolute nonrobustness in the presence of 
certain nonnormal distributions. 

The present Monte Carlo simulation demonstrates clearly that the robustness and nonrobustness of the 
normal theory correlation structure analytic techniques varies as a function of the data. The different pattern of 
results for the diagonal, block-diagonal, simplex, and circumplex models can be understood in the context of the 
work of Anderson and Amemiya (1985) and Browne and Shapiro (Browne, 1987; Browne & Shapiro, 1987), where 
they prove the asymptotic robustness of normal theory covariance structure analysis tests of models if the underlying 
correlation structure of the variables include orthogonal constructs. As such, theory predicts the large sample 
adequacy of the normal theory covariance structure analysis techniques for the diagonal and block-diagonal models, 
but not for simplex and circumplex models. In this Monte Carlo study, we see a similar pattern in the performance of 
the normal theory correlation structure analysis procedures as would be predicted for the covariance structure 
analysis procedures. 

The results show that when data are derived from a model with orthogonal variables, such as the diagonal 
and block-diagonal models, there is little functional difference in the asymptotic performance of the procedures 
under a wide variety of non-normal distributional conditions, and that by and large all the procedures asymptotically 
control empirical Type I error rates at the nominal level. In contrast, the results show that for models with non- 
orthogonal variables, such as the simplex and circumplex models, there is a substantial difference between the 
normal theory procedures and the asymptotically distribution free procedures. For these models, this study shows 
that the asymptotically distribution free procedures offer improved Type I error control over their normal theory 
counterparts. Though the normal theory procedures perform reasonably well for distributions with reduced marginal 
kurtoses and skew, this study reveals that the non-robustness of the normal theory statistical procedures can manifest 
for models of non-orthogonal variables with marginal distributions with ( K,S) as low as (1,1), (3,0), and (3, 1). 
Under these conditions and those of increased leptokurtosis and skew, the procedures which have been derived for 
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use under a wide variety of distributional conditions are the only procedures to control the empirical Type I error rate 
within a reasonable level of departure from the nominal level. 

The results of this paper show that when the null hypothesis that a model includes orthogonal variables is 
true, unless one has a particularly large sample size, it makes little difference whether one uses normal theory or 
alternative techniques; if a researcher does not have a large sample size, then the preferred procedures are the ones 
with the best performance under the given sample size conditions. However if the model does not include orthogonal 
variables and variables are substantially more leptokurtosed and/or skewed than ( K,S) of (1,0) and (-1,0), then 
procedures tierived for use under a wide variety of distributional conditions are preferred. This study provides 
evidence that the normal theory procedure with the best small sample Type I error control under conditions of 
extremely mild distributional non-normality was the normal theory quadratic form statistic correlation structure 
analysis statistic, NtR. The alternative structure analytic procedure with the best Type S I error control under more 
general non-normal distributional conditions was the asymptotically distribution free quadratic form correlation 
structure analysis test statistic, A dfR. 

Monte Carlo results in the early nineties evidenced that the standard asymptotically distribution free 
covariance structure analysis test statistic provides unacceptable Type I error control for all but the largest sample 
sizes (Chou, Bentler, & Satorra, 1991; Curran, West, & Finch, 1996; Henly, 1993; Hu, Bender, & Kano, 1992). The 
results of the present study shows that this is apparently not the case for the asymptotically distributional free 
quadratic form correlation structure analysis statistic derived from correlation distribution theory, AdfR , that AdfR 
shows the best overall performance under distributional nonnormality, and that instead of being liberal, it is 
consistently conservative. 



Implications in the context of current software availability 
The performance of general structure analytic techniques under conditions of multivariate nonnormality has 
been discussed for over a decade. Importantly, normal theory and asymptotically distribution free procedures are 
available for both correlation and covariance structure analysis techniques. At issue, however, is the relative 
performance of these procedures (c.f., Fouladi, in press). With the wide array of statistical options, researchers 
wanting to use structure analytic techniques need to be aware of how the different procedures perform, and whether 
there are procedures with better performance characteristics than the ones currently being recommended and used in 
the structure analytic research literature. 

A critical concern for the researcher is whether any of the currently available software packages can be used 
for the structure analysis of moderately non-normal data. At present, MULTICORR is the only program to 
implement the normal theory correlation structure analysis techniques discussed in this paper; there are no programs 
which implement the asymptotically distribution free correlation structure analysis techniques. However, with 
increasing sophistication of statistical software and the increasing ease of use of programming capabilities within 
these large scale software packages, users should be able to with a reasonable amount of effort implement the 
procedures discussed in the present paper. 
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Table 1: Summary statistics on the empirical rejection rates of NtR a NtZ . AdfR , and AdfZ across n conditions at nominal 
alpha=Q5 and .01. 



Statistic 








Alpha=.05 










Alpha=.01 






n 


Min 


Max 


Mean 


SE Mean 


Std Dev 


Min 


Max 


Mean 


SE Mean 


Std Dev 


NtR 


.340 


.0000 


.9998 


_ .1718 . 


.0134 


.2465 


.0000 


.9996 


.1074 


.0121 


.2225 


NtZ 


340 


.0352 " 


.9998 


.1907 


.0136 


.2506 


.0068 


.9996 


.1247 


.0125 


.2305 


AdfR 


321 


.0000 


.0672 


.0264 


.0010 


.0181 


.0000 


.0128 


.0031 


.0002 


.0033 


AdfZ 


321 


.0008 


1.0000 


.1333 


.0117 


.2088 


.0000 


1.0000 


.0866 


.0109 


.1948 



Table 2: Lower and upper limits of BRB guidelines for acceptable control of empirical Type I error rates. 




ALPHA 




«*■&« 


a±\a 


a±~a 


a±}a 


.05 


Lower limit 


.0450 


.0375 


.0250 


.0125 




Upper limit 


.0550 


.0625 


.0750 


.0875 


.01 


Lower limit 


.0090 


.0075 


.0050 


.0025 




Upper limit 


.0110 


.0125 


.0150 


.0175 



Table 3: Factorial Analysis of Variance (SS Type IV) results on percent bias (B) 



Eta - squared 

Effect dfgffeci df EtTOr NtR NtZ AdfR AdfZ 

1 482 .004 

2 482 .218 a 

12 482 .796 a 

4 482 .153 a 

3 482 .875 a 



Alpha 

P 

Distribution 

N:q 

Model 

Alpha x p 

Alpha x Distribution 

Alpha x N . 

Alpha x Model 
p x Distribution 
p x N 
p x Model 
Distribution x N 
Distribution x Model 
N x Model 



2 


482 


,020 b 


12 


482 


,065 b 


4 


482 


.004 


3 


482 


,122 a 


13 


482 


.004 


6 


482 


,030 c 


1 


482 


.179 a 


48 


482 


.411 a 


36 


482 


.877 a 


11 


482 


.335 a 



.010 c 


.607 a 


.003 


,222 a 


,069 a 


,049 a 


,876 a 


,312 a 


.127 a 


,069 a 


,763 a 


,420 a 


.925 a 


,222 a 


,489 a 


,027 b 


,124 a 


.000 


,072 a 


,409 a 


.007 


.000 


,794 a 


.002 


,149 a 


,152 a 


.015 


.003 


.013 


.004 


.020 


,106 a 


.178 a 


,202 a 


.007 


.001 


,376 a 


,298 a 


,192 a 


,928 a 


,126 a 


.475 a 


,355 a 


,236 a 


,480 a 



a=prob<.001 

b=prob<.01 

c=prob<.05 
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Table 4 



Summary fit (chi-sguare/df) for departure of empirical Type I error rate from the nominal level as a function of nominal 
alpha, p, model sample size, and distribution type. 









df... 


NtR 


NtZ- 


df . 


AdfR 


AdfZ 


Alpha 


.05 


340 


7382.6 


8118.3 


321 


92.3 


5300.9 






.01 


340 : 


26818.9 


30513.1 


321 


29.3 : 


22061.8 


p" 




2 "" 


130 


42.9 


16.4 


130 


58.0 


27.8 






6 


510 


15277.1 


17911.7 


484 


61.6 


15262.4 






12 


40 ' 


95790.1 


99940.6 


28 


60.5 - 


49742.1 


Model 


Diag 


280 


43.7 


95.3 


246 


60.4 


39.2 






Block 


130 


74.4 


207.1 


130 


71.0 


6473.1 






Simp 


130 


35196.0 


38131.7 


130 


53.5 


10732.6 






Circ 


140 


50222.3 


58028.5 


136 


58.8 « 


48066.4 


N 




2q 


136 


6967.7 


12053.0 


102 


148.5 : 


56298.8 


_ 




4q 


136 


12606.6 


15961.7 


132 


101.9 


19342.8 


— 




10q 


136 


20232.5 


21858.4 


136 


43.6 


3218.7 






20q 


136 


20756.8 


21507.2 


136 


22.8 


347.7 






50q 


136 


24940.1 


25198.4 


136 


10.4 


19.5 


Dist-Type Ku Sk 
















i 


-1 0 


50 


12.8 


21.3 


48 


40.8 


6434.9 




2. 


1 0 


50 


13.9 


52.7 


48 


46.5 


9628.4 




3 


1 1 


50 


218.7 


401.3 


48 


48.3 


5756.5 




4 


3 0 


50 


122.6 


254.4 


48 


53.3 


7280.5 




5 


3 1 


50 


430.9 


716.9 


48 


55.1 


6938.8 




6 


6 0 


50 


1444.3 


2094.1 


48 


61.6 


8329.9 




7 


6 1 


50 


2025.6 


2736.4 


48 


63.2 


8584.2 




8 


6 2 


50 


7303.2 


9691.9 


48 


61.9 


13506.9 




9 


25 0 


70 


49447.3 


56058.0 


62 


85.5 


32747.3 




10 


25 1 


50 


41070.0 


48460.5 


48 


86.3 


26617.6 




11 


25 2 


50 


42686.0 


49648.2 


48 


86.1 


26515.9 




12 


25 3 


60 


56566.2 


58285.1 


52 


51.1 


16002.5 




13 


het het 


50 


136.6 


193.1 


48 


44.7 


3759.7 


Overall 




680 


17100.8 


19315.7 


642 


60.8 


13681.3 
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Table 5: Minimum and maximum percent bias (EU) of empirical Type I error rates from the nominal level across n 
distribution types and percentaee of cells (%n)within the most liberal of the BRB guidelines as a function of nominal 
alpha. Model, p, and N:q 



Alpha Model 


p 








NtR 






NtZ 








AdfR 






AdfZ 




N:q 


n 


Min 


.Max 


%„ 


Min 


Max 


%„ 


n 


Min 


Max 


%„ 


Min 


Max 


%„ 


.05 Diag 


2 


2 


13 


-100 


o 

o 

t 


0 


-30 


i 


100 


13 


-97 


-92 


0 


-57 


-42 


100 






4 


13 


-32 


10 


100 


-8 


40 


100 


13 


-86 


-40 


77 


-78 


-16 


92 






10 


13 


-20 


31 


100 


-14 


36 


100 


13 


-69 


-9 


100 


-70 


0 


100 


” 




20 


13 


-8 


35 


100 


-4 


37 


100 


13 


-59 


4 


100 


-60 


8 


100 






50 


13 


-12 


27 


100 


-11 


28 


100 


13 


-32 


8 


100 


-32 


10 


100 




6 


2 


13 


-36 


18 


100 


16 


152 


77 




















4 


13 


-23 


52 


100 


02 


134 


77 


13 


-100 


-100 


0 


-98 


-24 


23 


• •- 




10 


13 


-12 


74 


100 


-2 


106 


77 


13 


-89 


-8 


77 


-90 


0 


77 






20 


13 


-20 


76 


92 


-16 


88 


77 


13 


-74 


14 


100 


-74 


17 


100 






50 


13 


-3 


56 


100 


-1 


60 


100 


13 


-58 


16 


100 


-58 


18 


100 




12 


2 


2 


-10 


62 


100 


32 


193 


50 




















4 


2 


6 


104 


50 


31 


172 


50 




















10 


2 


19 


97 


50 


30 


125 


50 


2 


-98 


-69 


50 


-97 


-68 


50 






20 


2 


18 


82 


50 


26 


92 


50 


2 


-72 


-18 


100 


-74 


-17 


100 






50 


2 


24 


59 


100 


27 


62 


100 


2 


-46 


-15 


100 


-46 


-14 


100 


Block 


6 


2 


13 


-52 


71 


100 


-18 


182 


31 


13 


-98 


-93 


0 


-68 


1314 


31 






4 


13 


-25 


112 


85 


-9 


138 


69 


13 


-88 


-61 


38 


-76 


738 


46 






10 


13 


-14 


100 


85 


-9 


112 


77 


13 


-78 


-16 


92 


-73 


149 


85 






20 


13 


-14 


90 


85 


-10 


94 


85 


13 


-66 


-9 


100 


-61 


15 


100 






50 


13 


-6 


54 


100 


-5 


55 


100 


13 


-48 


-2 


100 


-47 


1 


100 


Simp 


6 


2 


13 


16 


857 


31 


36 


1006 


23 


13 


-100 


-94 


0 


243 


1430 


0 






4 


13 


11 


1163 


23 


20 


1224 


23 


13 


-86 


-17 


77 


33 


710 


46 






10 


13 


-6 


1442 


23 


-4 


1461 


23 


13 


-67 


10 


100 


-6 


196 


69 






20 


13 


00 


1575 


23 


-2 


1584 


23 


13 


-52 


3' 


100 


-7 


86 


92 






50 


13 


04 


1690 


23 


-2 


1693 


23 


.13 


-42 


4 


100 


-18 


' 8 


100 


Circ 


6 


2 


12 


-1 


647 


33 


81 


1079 


0 


12 


-100 


-100 


0 


1096 


1850 


0 






4 


12 


3 


1018 


33 


40 


1228 


17 


12 


-90 


-55 


50 


200 


1511 


0 






10 


12 


-11 


1366 


17 


6 


1436 


17 


12 


-56 


17 


100 


44 


793 


42 






20 


12 


-10 


1534 


17 


-6 


1564 


17 


12 


-33 


34 


100 


20 


372 


75 






50 


12 


2 


1657 


17 


7 


1660 


17 


12 


-22 


33 


100 


-2 


94 


75 




12 


2 


2 


1476 


1632 


0 


1524 


1732 


0 


0 


















4 


2 


1727 


1821 


0 


1737 


1836 


0 


2 


-100 


-100 


0 


1882 


1900 


0 






10 


2 


1876 


1890 


0 


1878 


1892 


0 


2 


-78 


-16 


50 


879 


1141 


0 






20 


2 


1894 


1898 


0 


1894 


1898 


0 


2 


-65 


28 


100 


282 


434 


0 






50 


2 


1898 


1900 


0 


1898 


1900 


0 


2 


-49 


-12 


100 


-8 


90 


50 



continued 




14 



Correlation structure analysis 1 4 



Table 5 (continued!: Minimum and maximum percent bias (B^) of empirical Type I error rates from the nominal level 
across n distribution types and percentage of cells (%Jwithin the most liberal of the BRB ^guidelines as a function of 
nominal alpha. Model, p, and N:q 



Alpha 


Model 


P 








NtR 






NtZ 






AdfR 






AdfZ 




N:q 


n 


Min 


Max 


%n 


Min 


Max 


%n 


n 


Min 


Max 


%„ 


Min 


Max 


%n 


.01 


Diag 


2 


2 


13 


-100 


-100 


0 


-8 


70 


100 


13 


-100 


-100 


0 


-58 


-20 v 


100 








4 


13 


-100 


-84 


0 


18 


188 


62 


13 


-100 


-100 


0 


-98 


-38 


54 








10 


13 


-50 


118 


92 


-10 


204 


69 


13 


-98 


-54 


23 


-96 


-4 


69 








20 


13 


-18 


136 


77 


04 


184 


77 


13 


-96 


-26 


77 


-94 


-2 


77 








50 


13 


-20 


122 


77 


-18 


130 


77 


13 


-78 


-6 


85 


-78 


6 


85 






6 


2 


13 


-58 


56 


100 


54 


504 


15 






















4 


13 


-42 


136 


77 


22 


446 


46 


13 


-100 


-100 


0 


-100 


120 


0 


• — 






10 


13 


-12 


278 


77 


12 


382 


62 


13 


-100 


-42 


31 


-98 


-34 


54 








20 


13 


-52 


230 


69 


-24 


274 


69 


13 


-96 


06 


77 


-96 


16 


77 








50 


13 


-14 


194 


77 


-4 


210 


77 


13 


-84 


24 


85 


-86 


26 


85 






12 


2 


2 


18 


170 


50 


112 


646 


0 






















4 


2 


56 


318 


50 


112 


604 


0 






















10 


2 


100 


276 


0 


148 


380 


0 


2 


-100 


-100 


0 


-100 


-98 


0 








20 


2 


66 


238 


50 


74 


272 


50 


2 


-100 


-30 


50 


-100 


-30 


50 


1 






50 


2 


80 


160 


0 


84 


166 


0 


2 


-78 


-36 


50 


-78 


-34 


50 




Block 


6 


2 


13 


-84 


128 


62 


-32 


644 


15 


13 


-100 


-100 


0 


-80 


6170 


8 








4 


13 


-58 


346 


54 


-20 


492 


15 


13 


-100 


-84 


0 


-80 


3134 


31 








10 


13 


-14 


386 


31 


04 


438 


23 


13 


-98 


-36 


31 


-96 


520 


62 


- 






20 


13 


-32 


346 


38 


-26 


362 


38 


13 


-92 


-42 


69 


-82 


44 


77 








50 


13 


-12 


228 


54 


-10 


232 


54 


13 


-82 


-4 


92 


-82 


-4 


92 




Simp 


6 


2 


13 


26 


2862 


23 


98 


3892 


0 


13 


-100 


-100 


0 


792 


6750 


0 








4 


13 


22 


4392 


15 


40 


4942 


8 


13 


-100 


-68 


8 


42 


2818 


23 








10 


13 


-4 


6146 


15 


12 


6306 


15 


13 


-92 


-16 


77 


-22 


588 


69 








20 


13 


12 


7040 


15 


10 


7104 


15 


13 


-90 


2 


77 


-40 


208 


92 








50 


13 


0 


7896 


15 


8 


7882 


15 


13 


-58 


-4 


100 


-34 


12 


100 




Circ 


6 


2 


12 


-4 


1996 


25 


274 


4552 


O' 


12 


-100 


-100 


0 


4832 


9526 


0 








4 


12 


22 


3584 


17 


102 


5218 


0 


12 


-100 


-92 


0 


490 


7228 


0 








10 


12 


-8 


5642 


17 


16 


6274 


8 


12 


-94 


-24 


58 


62 


3182 


8 








20 


12 


-10 


6750 


17 


-2 


7032 


17 


12 


-82 


26 


92 


14 


1218 


58 








50 


12 


18 


7648 


17 


12 


7716 


17 


12 


-44 


28 


100 


-12 


216 


75 






12 


2 


2 


6360 


7452 


0 


6916 


8358 


0 






















4 


2 


8228 


9010 


0 


8348 


9238 


0 


2 


-100 


-100 


0 


9720 


9900 


0 








10 


2 


9600 


9726 


0 


9616 


9748 


0 


2 


-94 


-60 


50 


3368 


4606 


0 








20 


2 


9812 


9858 


0 


9818 


9860 


0 


2 


-84 


2 


50 


758 


1196 


0 








50 


2 


9890 


9896 


0 


9890 


9896 


0 


2 


-70 


-30 


100 


-30 


148 


50 





Correlation structure analysis 1 5 



§ 


































CM 




Tf 


00 


Tf 


Tf 


Ov 


vo 


CO 


CM 




vo 


00 








<N 




























»— i 




m 


in 


Tf 


CO 


m 


CM 


o 


CO 


CM 


m 


vo 


C-H 


































O 


CO 


CO 


OV 


r- 


r-* 


r- 


CO 


in 






i 


i 


o 




























































c 






Cy 


VO 


00 




CM 


Tf 


Ov 


CM 


m 


VO 


Tf 


VO 


CO 


r- 


CO 


00 


O 


VO 


o 


o 


CM 


VO 


VO 


CO 


O 


CM 


00 


o 








> 


Ov 


crv-ov 


Ov-Ov 


oo 


Ov 


oo 


CM 


CM 


CM 


oo 


OV 


CO 


CO 






CO 


r- 


m 




r- 


r- 


r- 


vo 


Tf 


o 








1 
















1 


i 


i 


1 


i 


Tf 


r- 




CO 








1 


i 


i 


i 


i 


1 


c 

a 




§ 




o 


CO 


o 


oo 


OV 




CO 




00 


CO 


O 






Tt- 


Ov 


oo 




oo 


vO 


Tf 


Tf 


CO 


CO 




oo 


Tf 




O 




CM 




CO 


CM 


VO 


vn 


Tf 


oo 


oo 


Ov 


CM 


i 


Ov 




CO 


CM 


CO 




CM 


»n 


in 


c- 


vO 


CM 


CO 


cd 

3 
























1 


1 


i 


i 








1 




1 


i 


i 


i 


i 


1 


i 


i 


* 


■* 






r- 


Tf 


o 


CM 


00 


vn 


Ov 


VO 


CO 


m 




CM 


o 


VO 


m 


CM 




O 


o 


CO 


vo 






Tf 


m 


CM 


C/3 






o 








»— ' 




CM 


CM 


CM 


r- 


vo 


r- 










CO 


CM 


Tf 


CO 


Tf 


CO 


VO 


m 


m 


CM 


CM 


O 






CM 


















1 


i 


i 


i 








1 


i 


l 


i 


1 


1 


i 


i 


i 


■ 


■ 


> 






<y 


r- 


r- 


m 


o 


Tf 


00 


00 


00 


00 


00 


Ov 




00 






CM 


O 


r- 


CM 


r» 


00 


r- 


00 


CM 


00 


r- 


c 

# o 

3 














1 




1 


1 


1 


in 

i 


1 


i 


i 






i 


1 


CM 

i 


i 


CM 

i 


CO 

1 


1 


Tf 

i 


CO 

1 


Tf 

i 


1 


i 


£ 






<y 




























CO 




Tf 


m 


m 


00 


r- 


Tf 


00 


00 


00 


Tf 


CO 


V- 

c/3 






CM 




























OV 

1 


Ov 

i 


Ov 

1 


OV 

i 


OV 

i 


Ov 

i 


Ov 

i 


OV 

1 


OV 

1 


OV 

1 


OV 

1 


Ov 

• 


OV 

1 


73 

O 








ooi- 


ooi- 

ooi- 

ooi- 


ooi- 


ooi- 


oor 


OOI- 


■100 


■100 


100 


OOI 


OOI 


-61 


-67 


-74 


-76 


m 

00 

i 


-82 


-83 


-83 


in 

oo 

1 


00 

oo 

1 


-87 


-75 


-61 


*3 






<y 


O Os 


m 


o 


Tf 


o 


vo 


ov 


Ov 


in 


00 


Ov 


00 


vO 


00 


VO 


Tf 


m 


, 


O 


m 


oo 


CO 


p 


Tf 


Tf 


s 




o 


p 


CM 


i 


Tf 


CO 

1 


vO 

i 


vo 

1 


Tf 

1 


00 

1 


oo 

1 


oo 

1 


CM 

i 


1 


1 


CO 

1 


i 


»n 

i 


VO 

i 


VO 

1 


VO 

1 


VO 

i 


r- 

i 


r- 

1 


o 

1 


CO 

1 


CO 

i 


C/3 


































































<y 


Tf 


Ov 


CM 


m 




vo 


o 


r- 




m 


’tt 






Ov 


VO 


r^- 


VO 


r- 


P 


o 


Ov 


vo 


CM 


m 


O 


Tf 








o 




1 




i— « 


1— * 


CM 


CO 


CM 


r- 


vo 


r- 




1 


i 


CM 


CO 


CO 


Tf 




wo 


CO 


vo 


m 


m 


CO 


CM 


- C/3 
C/3 






CM 








i 


1 


1 


1 


1 


i 


i 


1 


i 






1 


1 


1 


i 


l 




1 


i 


i 


i 


1 


i 


V-. 








WO 


Tf 


CO 


O 




O 


oo 


ov 


00 


00 


Ov 




vo 


CM 


00 


CM 




r- 




Ov 


O 


00 


00 


Tf 


Ov 


00 


cd 






8 














1 


1 


m 

i 


1 


’tt 

1 


i 




i 


1 


p 


CM 




CM 


CO 




Tf 


CO 


Tf 


1 


1 


m 


























































O 




























































C-H 






cy 


VO 


CO 


IT) 


r^* 


VO 


Ov 


o 




CM 


Ov 




CM 


CM 


O 


CM 


CM 


CO 


c- 


Tf 


Tf 


S 


CM 


CM 


OO 


VO 


oo 


W 






CM 


»— < 


CO 


CM 


»— < 


CM 


Tf 


CO 


vo 


in 


’tfr 


'tt 


m 


CM 


rf 


*—« 


m 


00 


CM 


CM 


CO 


OO 


o 


ov 




•— 4 












































*-* 
















1 








<y 


oc 


CM 


VO 


c- 




CM 


CO 


o 




CM 


oo 


’tt 


OO 




00 


r- 


o 


00 


VO 


in 


o 


Ov 


in 


00 


CM 


Ov 




















CM 


CM 




CO 


CO 


CM 


CO 




i 


CO 


xf 


CO 


in 


m 


vo 


00 


ov 


O 


CO 




1 


13 




























































c 

E 




N 


<y 


0 

2 


CM 


00 


VO 


Ov 


vo 


Tf 


O 


vo 


'tt 


CO 


vo 


00 


Tt* 


CO 


in 


r- 


Tf 


CO 


CO 


CM 


CM 




Ov 


Ov 




O 






i 








CM 


CM 


Ov 


o 


ov 








1 


CO 


CO 


Tf 


m 


in 


vo 




00 


Ov 




1 


o 

c 


















































*“ * 










o 








CM 


Tf 


CM 


O 


r- 


00 


o 


CM 




00 


ov 


vo 


VO 




CM 


CM 


CM 




Tf 




O 


Tf 


r- 


Tf 


Tf 


O 


Jm 






O 




• 












CM 


oo 


00 


r- 


CM 


»— < 


1 


1 


CM 


p— < 


Tf 


Tf 


CO 


m 


Ov 


00 


r- 


CM 










CM 


i 
























1 


























i 


s 




























































2 












O 


CM 


in 




CO 


o 


o 


oo 


CM 




O 


m 




O 


vO 


OO 


_ 4 


CM 


r- 


m 


00 


CM 


CM 




C/3 


2: 






O 

in 


i 










CM 






vo 


in 




CM 




i 




CM 






CO 


CM 


CM 


m 


Tf 


m 


CM 


i 


o 


C-H 




























































cd 


O 




























































Vn 

|_ 


C/3 






<y 




o 


CO 


VO 


CM 




VO 


CM 


00 


m 


o 


'tt 


CM 


m 


VO 


VO 


00 


m 


CO 


ov 


CO 




O 


vo 


OO 


CM 


o 

fc 


*c3 

> 

-E 






<N 


CO 


CM 

i 


CM 

i 


CO 

1 


CM 

1 


CM 

i 


CO 

1 


CM 

i 


, ” H 






CM 

i 


CO 

1 


’tfr 

i 


Tt 

1 


i 


CM 

i 








CM 


r- 


CM 


CM 


CM 

i 


m 

i 


<D 






























































HH 


im 






<y 


oo 


CO 


CM 


CO 


r-» 


CM 


oo 


m 


Tf 


CM 


o 


Ov 


»n 




o 


OO 


CM 


oo 


CM 


o 


Ov 


CM 


m 


CM 


«— 4 


m 


0) 


o 

e 










CM 






»— * 




1 




Tt 


in 




i 


*—* 


CM 


CM 




i 


CM 


CM 


CO 


Tf 


VO 


r- 


*— 4 




CM 


a 


cd 








1 


1 


' 


1 


1 


' 














1 


i 


< 






















i 


> 






























































H 


0 






<y 


VO 


Ov 


CM 


Tf 


CO 


Ov 


CM 


m 


CM 


Tf 


o 




CO 


O 


CO 


CM 


’tt 


VO 


Tf 


VO 


Tf 


3 


Tf 


CO 


Tf 


Tf 






o 


1 


i 


*— * 


1 


1 




*— H 




VO 




r^- 




1 






CM 


CM 


CO 


Tf 


Tf 


m 


r- 


00 






ed 








— *i 






i 
























1 






















1 


u 


£ 




























































;n 


<u 

V 








Ov O 


CO 


Tf 






t-* 


vo 


CM 


VO 


co 


00 


o 




CO 


vO 


r- 


r- 


CO 


CO 


r- 


O 


Tf 


O 


m 


Tf 


’5 


C/3 






o 






1 


1 










C- 


r- 


VO 




CM 


i 


1 






co 


Tf 


CO 


Tf 


Ov 


00 




CM 




E 

o 


Id 






CM 


1 


i 






















i 


























1 


C-H 


c 






<y 


CO 


CO 


r- 


O 


CM 




o 


CM 


VO 


m 


VO 


Ov 


00 


vo 


CM 


00 


Tf 


00 


O 




m 


Tf 


VO 


O 


CO 


CM 


o 


*3 






o 


1 


1 








CM 




i 


m 


m 


CO 






i 


1 








CO 


CM 


CM 


m 


Tf 


m 


CM 


i 


aj 


i 






in 






















































t-i 

3 


E 
























































V 






T3 




























































a 

o 

73 


c 

cd 








O O 


- 


O 


- 


O 


- 


CM 


O 


- 


CM 


CO 


het 


o 


o 


- 


o 


— 


O 


- 


CM 


O 


- 


CM 


CO 


het 


O 


s 




























































| 

c 


C/3 
' C/3 








p 


- 


- 


CO 


CO 


vo 


VO 


vo 


25 


25 


25 


25 


het 


1 


— 


- 


CO 


9 
z ' 


vo 


VO 


25 


25 


25 


25 


het 


o 


O 




























































p 


t: 




♦— 
























































Ui 

o 

a, 


3 




to 

Q 






CM 


CO 


Tf 


m 


vo 


r- 


00 


Ov 


o 




12 


CO 




CM 


CO 


Tf 


m 


vo 


r- 


00 


OV 


O 


1— i 


12 


CO 


<5 


Id 

c 


* 






hr 


















































O 

2 

H 


’3; 

c3 

E 




"<3 

1 




cd 

5 


























o 

O 

CQ 



























T 3 

O 

3 

C 

w 

c 

o 

a 



r- 

vH 



CO 

v—i 



O 

ERIC 



Table 6 (continued) Percentage departure of empirical Type I error rates from the nominal level of .05 across all simulated distribution types as a function of 
Model marginal kurtosis (Ku). marginal skew (Sk) and levels of N 
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Appendix A: Population correlation matrices & model matrices 
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Appendix B: Distribution types 

In the present study, data are generated using the method of Vale and Maurelli (1983). With this method, 
the possible combinations of skew and kurtosis are restricted; the maximum possible skew is a function of the 
kurtosis. The following table details the combinations of marginal skew and kurtosis considered. 
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Many of the recent Monte Carlo studies in covariance structure analysis have involved homogeneous 
marginal distributions. In the present study, the marginal distributions could be homogenous or heterogeneous. In the 
homogeneous univariate marginal condition, all the marginals were one of twelve types of distributions. In the 
heterogeneous univariate marginal condition, the sign of the skew, the level of skew, and the level of kurtosis of the 
marginals was varied. The heterogeneous marginal conditions ( K,S) were the following for the p - 2 model: (6,-2) and 
(6,-2), and for the p - 6 model: (6,-2), (6,-2), (25,1), (1,-1), (25,1), (6,-2). Heterogeneous marginal conditions were 
not simulated for p-Yl model. 
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Appendix C: Empirical type I error rates as afunction of p, model type, N:q, distribution type, and nominal 
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.0146 


.0000 


.0020 


- 








1 


.0400 


.0548 


.0166 


.0248 


.0000 


.0168 


.0000 


.0020 










2 


.0370 


.0512 


.0186 


.0254 


.0000 


.0166 


.0000 


.0028 








25 


0 


.0514 


.0680 


.0080 


.0112 


.0004 


.0282 


.0000 


.0002 










1 


.0552 


.0700 


.0094 


.0162 


.0008 


.0288 


.0000 


.0020 










2 


.0480 


.0630 


.0072 


.0136 


.0016 


.0242 


.0000 


.0012 










3 


.0388 


.0514 


.0188 


.0306 


.0002 


.0184 


.0000 


.0046 








hetero 


hetero 


.0338 


.0492 


.0294 


.0402 


.0000 


.0118 


.0000 


.0054 






10 


-1 


0 


.0402 


.0432 


.0400 


.0426 


.0054 


.0090 


.0024 


.0058 








1 - 


0 


.0446 


.0472 


.0408 


.0444 


.0062 


.0120 


.0018 


.0050 










1 


.0458 


.0512 


.0406 


. .0430 


.0050 


.0114 


.0026 


.0058 








3 


0 


.0504 


.0546 


.0342 


.0368 


.0080 


.0120 


.0028 


.0044 










1 


.0464 


.0488 


.0348 


.0370 


.0084 


.0128 


.0022 


.0036 








6 


0 


.0500 


.0550 


.0316 


.0344 


.0090 


.0134 


.0010 


.0018 










1 


.0508 


.0538 


.0292 


.0308 


.0096 


.0138 


.0010 


.0026 










2 


.0474 


.0514 


.0278 


.0308 


.0114 


.0176 


.0014 


.0034 








25 


0 


.0558 


.0604 


.0154 


.0158 


.0154 


.0236 


.0002 


.0004 










1 


.0584 


.0610 


.0160 


.0152 


.0156 


.0214 


.0004 


.0006 










2 


.0654 


.0680 


.0174 


.0158 


.0218 


.0304 


.0006 


.0010 










3 


.0502 


.0548 


.0376 


.0410 


.0078 


.0140 


.0018 


.0046 








hetero 


hetero 


.0472 


.0508 


.0456 


.0502 


.0068 


.0120 


.0046 


.0096 






20 


-1 


0 


.0520 


.0530 


.0502 


.0516 


.0082 


.0112 


.0074 


.0098 








1 


0 


.0574 


.0590 


.0520 


.0538 


.0100 


.0132 


.0060 


.0082 










1 


.0534 


.0558 


.0476 


.0506 


.0086 


.0112 


.0062 


.0066 








3 


0 


.0470 


.0500 


.0428 


.0464 


.0082 


.0118 


.0034 


.0046 










1 


.0526 


.0548 


.0460 


.0478 


.0108 


.0140 


.0032 


.0050 








6 


0 


.0504 


.0514 


.0378 


.0388 


.0110 


.0134 


.0036 


.0046 










1 


.0492 


.0506 


.0394 


.0412 


.0110 


.0132 


.0042 


.0046 










2 


.0470 


.0492 


.0466 


.0472 


.0114 


.0134 


.0048 


.0062 








25 


0 


.0590 


.0610 


.0204 


.0200 


.0192 


.0214 


.0012 


.0012 
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Appendix C continued: 



Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 



E 


Model 


N:q 


Ku 






Alpha= 


:.05 






Alpha= 


:.01 




Sk 


NtR 


NtZ 


AdfR 


AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


2 


Diag 


20 


25 


1 


.0594 


.0606 


.0230 


.0220 


.0192 


.0218 


.0004 


.0006 


■" 


* — 


- 




2 


' .0674 


.0684 


.0236 


.0244 


.0236 


.0284 


.0014 


.0012 










3 


.0526 


.0530 


.0454 


.0462 


.0110 


.0132 


.0050 


.0070 








hetero 


hetero 


.0462 


.0478 


.0460 


.0470 


.0086 


.0104 


.0068 


.0090 






50 


-1 


0 


.0558 


.0566 


.0542 


.0550 


.0114 


.0120 


.0094 


.0104 








1 


0 


.0532 


.0540 


.0532 


.0532 


.0090 


.0100 


.0090 


.0106 










1 


.0494 


.0504 


.0504 


.0508 


.0104 


.0108 


.0082 


.0088 








3 


0 


.0530 


.0530 


.0494 


.0500 


.0104 


.0114 


.0080 


.0086 










1 


.0472 


.0478 


.0450 


.0456 


.0088 


.0104 


.0066 


.0072 








6 


0 


.0440 


.0450 


.0416 


.0418 


.0084 


.0092 


.0068 


.0072 










1 


.0440 


.0444 


.0442 


.0442 


.0090 


.0098 


.0046 


.0050 










2 


.0474 


.0474 


.0468 


.0472 


.0124 


.0128 


.0088 


.0092 








25 


0 


.0580 


.0586 


.0366 


.0362 


.0180 


.0192 


.0028 


.0034 










1 


.0634 


.0640 


.0344 


.0346 


.0210 


.0230 


.0022 


.0022 










2 


.0620 


.0622 


.0338 


.0338 


.0222 


.0228 


.0024 


.0024 










3 


.0504 


.0510 


.0498 


.0508 


.0116 


.0128 


.0072 


.0080 








hetero 


hetero 


.0484 


.0490 


.0444 


.0450 


.0080 


.0082 


.0060 


.0068 


6 


Diag 


2 


-1 


0 


.0346 


.0578 






.0046 


.0154 












i 


0 


.0402 


.0664 






.0066 


.0202 














1 


.0384 


.0626 






.0052 


.0196 












3 


0 


.0320 


.0586 






.0042 


.0172 






- 








1 


.0388 


.0632 






.0084 


.0200 












6 


0 


.0380 


.0746 






.0070 


.0216 














1 


.0318 


.0650 






.0054 


.0194 














2 


.0390 


.0804 






.0068 


.0298 












25 


0 


.0592 


.1258 






.0152 


.0604 














1 


.0576 


.1244 






.0150 


.0600 














2 


.0550 


.1218 






.0156 


.0592 














3 


.0378 


.0758 






.0072 


.0276 












hetero 


hetero 


.0340 


.0612 


.0000 


.0000 


.0076 


.0182 


.0000 


.0000 






4 


-1 


0 


.0410 


.0540 


.0000 


.0018 


.0072 


.0140 


.0000 


.0002 








1 


0 


.0386 


.0512 


.0000 


.0008 


.0058 


.0122 


.0000 


.0002 










1 


.0442 


.0580 


.0000 


.0014 


.0064 


.0136 


.0000 


.0002 








3 


0 


.0436 


.0536 


.0000 


.0042 


.0098 


.0156 


.0000 


.0004 










1 


.0414 


.0534 


.0000 


.0028 


.0084 


.0156 


.0000 


.0002 








6 


0 


.0440 


.0612 


.0000 


.0054 


.0112 


.0198 


.0000 


.0014 










1 


.0458 


.0614 


.0000 


.0042 


.0104 


.0188 


.0000 


.0008 










2 


.0524 


.0702 


.0000 


.0076 


.0128 


.0238 


.0000 


.0018 








25 


0 


.0718 


.1168 


.0000 


.0372 


.0230 


.0514 


.0000 


.0200 










1 


.0760 


.1160 


.0000 


.0378 


.0206 


.0472 


.0000 


.0220 










2 


.0698 


.1142 


.0000 


.0368 


.0236 


.0546 


.0000 


.0202 










3 


.0454 


.0668 


.0000 


.0086 


.0102 


.0222 


.0000 


.0022 








hetero 


hetero 


.0426 


.0542 


.0000 


.0016 


.0066 


.0126 


.0000 


.0000 






10 


-1 


0 


.0472 


.0502 


.0452 


.0498 


.0088 


.0114 


.0048 


.0066 








1 


0 


.0456 


.0508 


.0354 


.0384 


.0090 


.0112 


.0030 


.0034 










1 


.0442 


.0488 


.0426 


.0452 


.0106 


.0134 


.0046 


.0056 




' ■ 




3 


0 


.0480 


.0542 


.0302 


.0308 


.0112 


.0134 


.0022 


.0026 










1 


.0484 


.0532 


.0328 


.0356 


.0110 


.0138 


.0022 


.0024 
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Appendix C continued: 



Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 



p 


Model 


N:q 


Ku 






Alpha= 


:.05 






Alpha=.01 




Sk 


NtR 


NtZ 


AdfR 


AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


6 


Diag 


10 


6 


0 


.0544 


.0596 


.0200 


.0194 


.0114 


.0156 


.0004 


.0008 




— 


- 




1 


' .0562 


.0630 


.0220 


.0236 


.0144 


.0184 


.0006 


.0010 










2 


.0526 


.0622 


.0256 


.0280 


.0118 


.0164 


.0022 


.0028 








25 


0 


.0810 


.0950 


.0056 


.0058 


.0306 


.0408 


.0000 


.0002 










1 


.0868 


.1028 


.0074 


.0084 


.0362 


.0478 


.0002 


.0004 










2 


.0852 


.0972 


.0058 


.0048 


.0378 


.0482 


.0000 


.0002 










3 


.0504 


.0566 


.0354 


.0382 


.0152 


.0186 


.0020 


.0026 








hetero 


hetero 


.0484 


.0530 


.0458 


.0480 


.0106 


.0124 


.0058 


.0066 






20 


-1 


0 


.0406 


.0440 


.0568 


.0586 


.0048 


.0076 


.0106 


.0116 








1 


0 


.0448 


.0480 


.0456 


.0482 


.0082 


.0094 


.0100 


.0112 










1 


.0484 


.0512 


.0488 


.0502 


.0100 


.0104 


.0058 


.0070 








3 


0 


.0482 


.0500 


.0424 


.0438 


.0070 


.0094 


.0054 


.0054 










1 


.0504 


.0534 


.0444 


.0458 


.0108 


.0120 


.0058 


.0066 








6 


0 


.0554 


.0588 


.0370 


.0374 


.0116 


.0144 


.0046 


.0054 










1 


.0534 


.0552 


.0348 


.0354 


.0140 


.0148 


.0030 


.0030 










2 


.0580 


.0610 


.0364 


.0372 


.0156 


.0174 


.0038 


.0044 








25 


0 


.0862 


.0920 


.0146 


.0136 


.0330 


.0374 


.0008 


.0006 










1 


.0878 


.0938 


.0174 


.0176 


.0316 


.0368 


.0004 


.0004 










2 


.0816 


.0896 


.0128 


.0130 


.0310 


.0370 


.0004 


.0006 


- 








3 


.0588 


.0630 


.0432 


.0442 


.0178 


.0206 


.0064 


.0066 








hetero 


hetero 


.0400 


.0420 


.0478 


.0500 


.0090 


.0100 


.0074 


.0078 






50 


-1 


0 


.0484 


.0496 


.0526 


.0536 


.0120 


.0124 


.0092 


.0094 








1 


0 


.0486 


.0504 


.0522 


.0536 


.0090 


.0096 


.0076 


.0082 










1 


.0534 


.0550 


.0516 


.0526 


.0114 


.0118 


.0092 


.0094 








3 


0 


.0502 


.0510 


.0450 


.0452 


.0096 


.0100 


.0082 


.0088 










1 


.0512 


.0526 


.0506 


.0518 


.0126 


.0128 


.0076 


.0078 








6 


0 


.0604 


.0618 


.0452 


.0462 


.0126 


.0132 


.0070 


.0072 










1 


.0548 


.0564 


.0458 


.0462 


.0108 


.0116 


.0058 


.0060 










2 


.0488 


.0502 


.0456 


.0462 


.0086 


.0100 


.0080 


.0082 








25 


0 


.0782 


.0800 


.0210 


.0208 


.0294 


.0310 


.0016 


.0014 










1 


.0776 


.0790 


.0260 


.0260 


.0282 


.0298 


.0026 


.0028 










2 


.0680 


.0708 


.0256 


.0256 


.0240 


.0260 


.0016 


.0016 










3 


.0596 


.0604 


.0478 


.0480 


.0170 


.0174 


.0054 


.0056 








hetero 


hetero 


.0542 


.0548 


.0578 


• .0588 


.0110 


.0116 


.0124 


.0126 


6 


Block 


2 


-1 


0 


.0274 


.0698 


.0036 


.5560 


.0022 


.0238 


.0000 


.4644 








1 


0 


.0270 


.1062 


.0030 


.7072 


.0024 


.0528 


.0000 


.6270 










1 


.0422 


.0760 


.0032 


.2272 


.0062 


.0330 


.0000 


.1440 








3 


0 


.0362 


.0914 


.0026 


.5288 


.0048 


.0454 


.0000 


.4334 










1 


.0574 


.1134 


.0024 


.4222 


.0098 


.0582 


.0000 


.3222 








6 


0 


.0564 


.1118 


.0008 


.4168 


.0100 


.0566 


.0000 


.3156 










1 


.0546 


.1172 


.0016 


.4296 


.0092 


.0544 


.0000 


.3290 










2 


.0616 


.1022 


.0028 


.2132 


.0152 


.0488 


.0000 


.1514 








25 


0 


.0856 


.1410 


.0010 


.3014 


.0228 


.0744 


.0000 


.2362 










1 


.0600 


.1010 


.0010 


.0660 


.0102 


.0448 


.0000 


.0424 










2 


.0630 


.0990 


.0012 


.0604 


.0172 


.0422 


.0000 


.0390 










3 


.0358 


.0530 


.0032 


.0220 


.0024 


.0150 


.0000 


.0092 




- ■ 




hetero 


hetero 


.0242 


.0410 


.0036 


.0160 


.0016 


.0068 


.0000 


.0020 






4 


-1 


0 


.0380 


.0482 


.0196 


.2664 


.0042 


.0100 


.0008 


.1812 
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Appendix C continued: 



Empirical Type 1 error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 



p 


Model 


N:q 


Ku 


Sk 




Alpha= 05 






Alpha=.01 




NtR 


NtZ 


AdfR 


AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


6 


Block 


4 


1 


0 


.0402 


.0692 


.0166 


.4190 


.0064 


.0234 


.0004 


.3234 










1 


' .0588 


.0734 


.0132 


.0498 


.0170 


.0266 


.0006 


'.0146 








3 


0 


.0490 


.0652 


.0122 


.2230 


.0130 


.0258 


.0004 


.1400 










1 


.0640 


.0788 


.0074 


.1150 


.0212 


.0370 


.0004 


.0570 








6 


0 


.0608 


.0778 


.0088 


.1350 


.0170 


.0300 


.0000 


.0748 










1 


.0652 


.0824 


.0086 


.1258 


.0216 


.0342 


.0002 


.0662 










2 


.0746 


.0902 


.0086 


.0420 


.0276 


.0408 


.0002 


.0184 








25 


0 


.0812 


.0996 


.0076 


.0122 


.0298 


.0424 


.0002 


.0036 










1 


.0876 


.1026 


.0060 


.0134 


.0344 


.0470 


.0000 


.0036 










2 


.1058 


.1192 


.0064 


.0152 


.0446 


.0592 


.0000 


.0046 










3 


.0506 


.0562 


.0126 


.0188 


.0128 


.0180 


.0002 


.0020 








hetero 


hetero 


.0376 


.0456 


.0194 


.0262 


.0048 


.0080 


.0016 


.0022 






10 


-1 


0 


.0548 


.0592 


.0420 


.0972 


.0094 


.0112 


.0064 


.0364 








1 


0 


.0436 


.0482 


.0308 


.1246 


.0088 


.0112 


.0030 


.0620 










1 


.0612 


.0666 


.0272 


.0310 


.0190 


.0222 


.0026 


.0034 








3 


0 


.0618 


.0676 


.0230 


.0622 


.0170 


.0208 


.0018 


.0212 










1 


.0682 


.0734 


.0176 


.0308 


.0246 


.0270 


.0018 


.0052 








6 


0 


.0722 


.0772 


.0196 


.0420 


.0264 


.0302 


.0020 


.0118 










1 


.0728 


.0764 


.0202 


.0380 


.0240 


.0282 


.0010 


.0086 










2 


.0768 


.0814 


.0176 


.0228 


.0312 


.0350 


.0020 


.0026 








25 


0 


.0998 


.1060 


.0108 


.0236 


.0486 


.0538 


.0004 


.0046 










1 


.0872 


.0910 


:0136 


.0134 


.0378 


.0416 


.0002 


.0004 










2 


.0914 


.0954 


.0180 


.0182 


.0382 


.0434 


.0008 


.0008 










3 


.0572 


.0596 


.0328 


.0358 


.0198 


.0212 


.0020 


.0032 








hetero 


hetero 


.0430 


.0454 


.0328 


.0332 


.0086 


.0104 


.0050 


.0054 






20 


-1 


0 


.0480 


.0482 


.0456 


.0530 


.0084 


.0088 


.0044 


.0104 








1 


0 


.0486 


.0488 


.0372 


.0576 


.0106 


.0122 


.0056 


.0144 










1 


.0582 


.0608 


.0316 


.0338 


.0166 


.0174 


.0028 


.0032 








3 


0 


.0534 


.0562 


.0318 


.0396 


.0142 


.0162 


.0032 


.0054 










1 


.0684 


.0704 


.0266 


.0300 


.0248 


.0262 


.0040 


.0048 








6 


0 


.0714 


.0722 


.0282 


.0352 


.0224 


.0232 


.0026 


.0056 










1 


.0666 


.0686 


.0248 


.0286 


.0238 


.0252 


.0010 


.0028 










2 


.0736 


.0750 


.0304 


.0318 


.0262 


.0278 


.0040 


.0040 








25 


0 


.0952 


.0972 


.0168 


.0196 


.0446 


.0462 


.0008 


.0018 










1 


.0920 


.0936 


.0240 


.0244 


.0386 


.0404 


.0022 


.0022 










2 


.0852 


.0870 


.0224 


.0230 


.0312 


.0334 


.0022 


.0022 










3 


.0624 


.0620 


.0348 


.0374 


.0224 


.0236 


.0058 


.0064 








hetero 


hetero 


.0432 


.0450 


.0378 


.0390 


.0068 


.0074 


.0054 


.0062 






50 


-1 


0 


.0470 


.0474 


.0488 


.0504 


.0104 


.0104 


.0096 


.0096 








1 


0 


.0490 


.0506 


.0460 


.0480 


.0112 


.0116 


.0066 


.0070 










1 


.0590 


.0600 


.0440 


.0442 


.0162 


.0168 


.0072 


.0076 








3 


0 


.0572 


.0578 


.0394 


.0400 


.0138 


.0144 


.0046 


.0046 










1 


.0592 


.0592 


.0416 


.0416 


.0166 


.0170 


.0056 


.0060 








6 


0 


.0650 


.0654 


.0394 


.0390 


.0164 


.0168 


.0046 


.0048 










1 


.0604 


.0612 


.0306 


.0316 


.0204 


.0208 


.0052 


.0062 










2 


.0626 


.0634 


.0450 


.0458 


.0228 


.0234 


.0072 


.0070 




" ■ 




25 


0 


.0772 


.0774 


.0258 


.0264 


.0328 


.0332 


.0018 


.0018 










1 


.0728 


.0738 


.0310 


.0312 


.0268 


.0276 


.0034 


.0034 
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Appendix C continued: 

Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 
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.0492 
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2 


-1 


0 


.0590 


.0716 


.0028 


.1750 


.0126 


.0198 
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.0894 






1 


0 


.0580 


.0682 


.0022 


.1714 


.0134 


.0202 
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.0892 








1 


.1258 


.1492 


.0016 


.2824 


.0390 


.0542 


.0000 


.1748 






3 


0 


.0868 


.1062 


.0012 


.2194 


.0262 


.0392 


.0000 


.1246 








1 


.1180 


.1428 


.0006 


.2620 


.0374 


.0578 


.0000 


.1600 






6 


0 


.1258 


.1550 


.0006 


.3016 


.0450 


.0696 


.0000 


.1846 








1 


.1454 


.1862 


.0014 


.3380 


.0582 


.0886 


.0000 


.2210 








2 


.3510 


.4052 


.0002 


.5718 


.1656 


.2290 
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.4630 






25 


0 


.4688 


.5532 


.0008 


.7648 


.2790 


.3944 
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.6850 








1 


.4478 


.5360 


.0004 


.7562 


.2722 


.3874 


.0000 


.6748 








2 


.4784 


.5524 


.0006 


.7526 


.2962 


.3992 
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.6692 
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.3960 


.4410 


.0012 


.5656 


.2110 


.2832 
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.4680 






hetero 
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.0648 


.0758 


.0024 


.1750 


.0158 


.0240 
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.0896 
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-1 


0 


.0556 


.0598 


.0414 


.0786 


.0122 


.0140 


.0032 


.0176 






1 


0 


.0632 


.0664 


.0312 


.0664 


.0158 


.0192 


.0010 


.0142 








1 


.1284 


.1442 


.0272 


.0854 


.0378 


.0474 


.0014 


.0214 






3 


0 


.0986 


.1054 


.0262 


.0712 


.0308 


.0374 


.0010 


.0156 








1 


.1330 


.1466 


.0276 


.0826 


.0428 


.0526 


.0004 


.0204 






6 


0 


.1618 


.1870 


.0226 


.0908 


.0570 


.0758 


.0010 


.0272 








1 


.1778 


.2022 


.0182 


.0916 


.0734 


.0922 


.0002 


.0322 








2 


.3886 


.4176 


.0168 


.1562 


.1950 


.2336 


.0004 


.0656 






25 


0 


.5982 


.6332 


.0070 


.3942 


.4188 


.4746 


.0000 


.2804 








1 


.6210 


.6508 


.0096 


.3978 


.4398 


.4930 
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.2860 








2 
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.6618 


.0094 


.4052 


.4492 


.5042 


.0000 


.2918 








3 


.5164 


.5396 


.0254 


.2700 


.3424 


.3846 


.0016 


.1692 
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.0726 


.0772 


.0286 


.0742 


.0180 


.0232 


.0022 


.0174 




10 


-1 


0 


.0468 


.0480 


.0472 


.0590 


.0096 


.0112 


.0080 


.0144 






1 


0 
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.0588 


.0552 


.0638 


.0128 


.0150 


.0084 


.0130 








1 


.1326 


.1354 
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.0602 


.0374 


.0420 


.0064 


.0122 






3 


0 


.0992 


.1026 


.0414 


.0502 


.0318 


.0318 


.0052 


.0096 








1 


.1572 


.1686 


.0402 


.0582 


.0544 


.0598 


.0052 


.0100 






6 


0 


.2138 


.2264 


.0324 


.0472 


.0970 


.1052 


.0026 


.0078 








1 


.2552 


.2702 


.0342 


.0566 


.1098 


.1220 


.0036 


.0086 








2 


.4356 


.4390 


.0310 


.0574 


.2420 


.2602 


.0028 


.0118 






25 


0 


.7536 


.7628 


.0166 


.1208 


.6016 


.6216 


.0008 


.0558 








1 


.7546 


.7658 


.0180 


.1302 


.5982 


.6246 


.0008 


.0582 








2 


.7712 


.7806 


.0170 


.1352 


.6246 


.6406 


.0010 


.0626 








3 


.6948 


.6962 


.0416 


.1480 


.5436 


.5546 


.0062 


.0688 
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hetero 


.0638 


.0674 


.0438 


.0582 


.0188 


.0194 


.0048 


.0100 




20 


-1 


0 


.0502 


.0490 


.0492 


.0554 


.0112 


.0110 


.0096 


.0126 






1 


0 


.0630 


.0628 


.0504 


.0548 


.0130 


.0142 


.0102 


.0132 








1 


.1498 


.1502 


.0514 


.0584 


.0490 


.0492 


.0094 


.0126 






3 


0 


.1090 


.1114 


.0418 


.0464 


.0312 


.0332 


.0072 


.0096 


" 






1 


.1762 


.1798 


.0454 


.0516 


.0638 


.0676 


.0076 


.0102 






6 


0 


.2552 


.2602 


.0402 


.0486 


.1158 


.1230 


.0068 


.0110 
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Appendix C continued: 



Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 
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Alpha=.01 
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NtZ 
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AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


6 


Simp 


20 


6 


1 


.2730 


.2774 


.0388 


.0468 


.1302 


.1346 


.0040 


.0060 








* ~ 


2 


.4638 


.4694 


.0442 


.0606 


.2678 


.2688 


.0054 


.0114 








25 


0 


.8300 


.8330 


.0266 


.0694 


.7112 


.7180 


.0010 


.0148 










1 


.8288 


.8322 


.0238 


.0672 


.7010 


.7084 


.0016 


.0164 










2 


.8376 


.8420 


.0270 


.0672 


.7140 


.7204 


.0018 


.0160 










3 


.8100 


.8100 


.0460 


.0928 


.6752 


.6780 


.0064 


.0308 








hetero 


hetero 


.0670 


.0676 


.0400 


.0464 


.0176 


.0196 


.0080 


.0106 






50 


-1 


0 


.0520 


.0492 


.0520 


.0536 


.0100 


.0108 


.0092 


.0108 








1 


0 


.0602 


.0590 


.0484 


.0494 


.0120 


.0114 


.0096 


.0102 










1 


.1364 


.1358 


.0474 


.0510 


.0452 


.0454 


.0090 


.0100 








3 


0 


.1278 


.1302 


.0500 


.0526 


.0394 


.0402 


.0066 


.0078 










1 


.1758 


.1778 


.0430 


.0462 


.0594 


.0618 


.0082 


.0104 








6 


0 


.2796 


.2808 


.0460 


.0462 


.1342 


.1354 


.0064 


.0074 










1 


.3228 


.3266 


.0488 


.0492 


.1608 


.1642 


.0080 


.0082 










2 


.4944 


.4954 


.0478 


.0528 


.2922 


.2946 


.0094 


.0112 








25 


0 


.8818 


.8812 


.0314 


.0442 


.7742 


.7770 


.0044 


.0066 










1 


.8792 


.8810 


.0292 


.0412 


.7720 


.7742 


.0046 


.0070 










2 


.8952 


.8964 


.0330 


.0458 


.7910 


.7908 


.0042 


.0078 










3 


.8920 


.8918 


.0340 


.0514 


.7996 


.7982 


.0066 


.0112 


- 
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hetero 


.0742 


.0748 


.0508 


.0540 


.0202 


.0204 


.0092 


.0106 
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2 


-1 


0 


.0496 


.0984 
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.5980 


.0096 


.0374 
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.4932 








1 


0 


.0502 


.0906 


.0000 


.5998 


.0130 


.0390 


.0000 


.4944 










1 


.0972 


.1790 


.0000 


.7122 


.0298 


.0854 


.0000 


.6224 








3 


0 


.0788 


.1470 


.0000 


.6774 


.0190 


.0720 


.0000 


.5820 










1 


.0958 


.1764 


.0000 


.7166 


.0276 


.0854 


.0000 


.6280 








6 


0 


.1008 


.2050 


.0000 


.7702 


.0262 


.1164 


.0000 


.6920 










1 


.1150 


.2144 


.0000 


.7638 


.0378 


.1192 


.0000 


.6902 










2 


.2634 


.4072 


.0000 


.8916 


.1092 


.2586 


.0000 


.8484 








25 


0 


.3632 


.5822 


.0000 


.9728 


.2026 


.4592 


.0000 


.9596 










1 


.3728 


.5894 


.0000 


.9750 


.2096 


.4610 


.0000 


.9626 










2 


.3736 


.5850 


.0000 


.9732 


.2066 


.4652 


.0000 


.9612 










3 


.7368 


.7830 


.0000 


.8802 


.5422 


.6368 


.0000 


.8238 
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hetero 


.0638 


.1206 


.0000 


.6302 


.0160 


.0516 


.0000 


.5258 
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-1 


0 


.0530 


.0698 


.0224 


.1498 


.0122 


.0202 


.0008 


.0590 








1 


0 


.0516 


.0802 


.0196 


.1528 


.0134 


.0242 


.0002 


.0676 










1 


.1112 


.1526 


.0140 


.2122 


.0338 


.0632 


.0006 


.1110 








3 


0 


.0766 


.1176 


.0148 


.1928 


.0224 


.0458 


.0000 


.0950 










1 


.1206 


.1694 


.0162 


.2362 


.0406 


.0708 


.0000 


.1302 








6 


0 


.1528 


.2220 


.0094 


.3010 


.0596 


.1170 


.0000 


.1868 










1 


.1598 


.2306 


.0102 


.2950 


.0644 


.1186 


.0000 


.1900 










2 


.3258 


.4114 


.0048 


.4532 


.1564 


.2564 


.0000 


.3262 








25 


0 


.5554 


.6622 


.0082 


.8056 


.3638 


.5242 


.0000 


.7328 










1 


.5504 


.6642 


.0106 


.8028 


.3662 


.5318 


.0000 


.7298 










2 


.5592 


.6638 


.0084 


.7970 


.3684 


.5206 


.0000 


.7262 










3 


.9622 


.9658 


.0910 


.7986 


.8968 


.9112 


.0016 


.6586 
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.1124 
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.1796 


.0238 
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.0006 


.0836 
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-1 


0 


.0444 


.0530 


.0488 


.0778 


.0092 


.0116 


.0070 


.0186 








1 


0 


.0574 


.0670 


.0442 


.0718 


.0136 


.0184 


.0052 


.0162 
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Appendix C continued: 



Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 



E 


Model 


N:q 


Ku 


Sk 




Alpha= 


.05 






Alpha=.01 




NtR 


NtZ 


AdfR 


AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


6 


Circ 


10 


1 


1 


.1240 


.1374 


.0428 


.0830 


.0374 


.0492 


.0058 


.0246 








_ 3 


0 
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.0308 
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.0216 
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.1494 


.1716 


.0354 


.0816 


.0528 


.0714 


.0020 


.0198 








6 


0 
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.2444 


.0332 


.0970 


.0900 


.1236 


.0036 


.0334 




■- 




6 


1 


.2360 


.2730 


.0368 


.1044 


.1106 


.1330 


.0056 


.0388 








6 


2 


.3908 


.4338 


.0276 


.1426 


.2122 


.2556 


.0022 


.0640 








25 


0 


.7228 


.7594 


.0256 


.4410 


.5586 


.6206 


.0022 


.3226 








25 


1 


.7300 


.7674 


.0218 


.4418 


.5632 


.6316 


.0006 


.3258 








25 


2 


.7330 


.7678 


.0290 


.4464 


.5742 


.6374 


.0010 


.3282 








25 


3 


.9996 


.9994 


.8362 


.9684 


.9986 


.9984 


.4776 


.8752 








hetero 


hetero 


.1108 


.1222 


.0584 


.0886 


.0406 


.0490 


.0076 


.0254 






20 


-1 


0 


.0450 


.0470 


.0472 


.0598 


.0090 


.0098 


.0098 


.0158 








1 


0 


.0554 


.0574 


.0500 


.0610 


.0122 


.0142 


.0108 


.0160 










1 


.1278 


.1366 


.0452 


.0632 


.0384 


.0434 


.0074 


.0146 








3 


0 


.1156 


.1254 


.0482 


.0600 


.0318 


.0376 


.0054 


.0118 










1 


.1772 


.1954 


.0436 


.0644 


.0634 


.0754 


.0066 


.0114 








6 


0 


.2716 


.2942 


.0392 


.0666 


.1248 


.1454 


.0052 


.0128 










1 


.2990 


.3220 


.0372 


.0658 


.1462 


.1642 


.0050 


.0140 










2 


.4424 


.4608 


.0408 


.0796 


.2432 


.2698 


.0038 


.0198 








25 


0 


.7992 


.8158 


.0334 


.2278 


.6724 


.6962 


.0028 


.1286 










1 


.8134 


.8318 


.0356 


.2304 


.6826 


.7082 


.0034 


.1268 










2 


.8172 


.8284 


.0340 


.2360 


.6850 


.7132 


.0018 


.1318 










3 


1.0000 


1.0000 


.9908 


.9992 


1.0000 


1.0000 


.9474 


.9896 








hetero 


hetero 


.1358 


.1422 
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.0830 


.0558 


.0592 


.0126 


.0198 






50 


-1 


0 


.0512 


.0534 


.0542 


.0578 . 


.0118 


.0112 


.0106 


.0134 
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0 


.0548 


.0582 


.0474 


.0502 


.0144 


.0160 


.0122 


.0140 










1 


.1202 


.1222 


.0444 


.0488 


.0344 


.0380 


.0080 


.0094 








3 


0 


.1302 


.1362 


.0492 


.0534 


.0406 


.0460 


.0096 


.0116 










1 


.1876 


.1928 


.0460 


.0516 


.0710 


.0742 


.0094 


.0102 








6 


0 


.3106 


.3214 


.0470 


.0518 


.1504 


.1616 


.0090 


.0088 










1 


.3540 


.3672 


.0440 


.0514 


.1836 


.1934 


.0068 


.0090 










2 


.4658 


.4722 


.0422 


.0540 


.2694 


.2816 


.0070 


.0106 








25 


0 


.8784 


.8802 


.0456 


.0970 


.7726 


.7792 


.0056 


.0298 










1 


.8738 


.8772 


.0388 


.0888 


.7642 


.7756 


.0060 


.0266 










2 


.8780 


.8796 


.0406 


.0936 


.7748 


.7816 


.0070 


.0316 










3 


1.0000 


1.0000 


1.0000 


1.0000 


1.0000 


1.0000 


.9994 


1.0000 








hetero 


hetero 


.1612 


.1634 


.0664 


.0712 


.0668 


.0668 


.0128 


.0146 
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2 


25 


0 


.0810 


.1464 






.0270 


.0746 














3 


.0450 


.0662 






.0118 


.0212 










4 


25 


0 


.1018 


.1362 






.0418 


.0704 














3 


.0532 


.0654 






.0156 


.0212 










10 


25 


0 


.0986 


.1124 


.0008 


.0014 


.0376 


.0480 


.0000 


.0000 










3 


.0594 


.0648 


.0154 


.0162 


.0200 


.0248 


.0000 


.0002 






20 


25 


0 


.0910 


.0958 


.0140 


.0132 


.0338 


.0372 


.0000 


.0000 










3 


.0590 


.0628 


.0412 


.0416 


.0166 


.0174 


.0070 


.0070 






50 


25 


0 


.0794 


.0810 


.0270 


.0272 


.0260 


.0266 


.0022 


.0022 




' ■ 




25 


3 


.0622 


.0636 


.0424 


.0430 
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.0184 


.0064 


.0066 


12 
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2 


25 


0 


.8658 


.9160 






.7552 


.8458 







0 
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Appendix C continued: 



Empirical Type I error rates as a function of p, model type, N:q, Kurtosis (Ku), Skew (Sk), and nominal alpha. 
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Alpha=.05 
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NtR 


NtZ 


AdfR 


AdfZ 


NtR 


NtZ 


AdfR 


AdfZ 


12 


Circ 


2 


25 


3 


.7880 


.8120 






.6460 


.7016 






— ’ 


• - 


4 


- '25 •' 


0 


.9604 


.9678 


.0000 


1.0000 


.9110 


.9338 


.0000 


1.0000 






4 


25 


3 


.9134 


.9186 


.0000 


.9910 


.8328 


.8448 


.0000 


.9820 






10 


25 


0 


.9950 


.9960 


.0110 


.6206 


.9826 


.9848 


.0006 


.4706 






10 


25 


3 


.9880 


.9888 


.0420 


.4896 


.9700 


.9716 


.0040 


.3468 






20 


25 


0 


.9990 


.9990 


.0174 


.1908 


.9958 


.9960 


.0016 


.0858 






20 


25 


3 


.9970 


.9972 


.0642 


.2672 


.9912 


.9918 


.0102 


.1296 






50 


25 


0 


.9998 


.9998 


.0254 


.0460 


.9996 


.9996 


.0030 


.0070 






50 


25 


3 


.9992 


.9992 


.0440 


.0948 


.9990 


.9990 


.0070 


.0248 
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